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PURPOSE  OF  THE  REPORT:  To  provide  a  detailed  examination  of  the  impacts  and  issues  associated 
with  involving  soldiers  in  early  equipment  prototyping.  Based  on  this  examination,  this  paper  provides 
recommendations  for  better  utilization  of  soldiers  in  the  future. 

ABSTRACT:  Early  user  involvement  has  many  known  benefits  and  has  been  used  by  civilian  industry  to 
find  problems  in  products  before  the  products  are  fully  distributed.  Early  soldier  involvement  also 
provides  benefits  that  can  lead  to  significant  cost  avoidance,  but  has  additional  complex  political  and 
economic  ramifications  that  must  be  addressed.  Areas  of  examination  included  impacts  of  differing 
expectations  by  affected  stakeholders  (Users,  DOD,  ARSTAF,  Congress),  impacts  on  the  final 
configuration  design,  impacts  on  soldier  training,  and  impacts  if  prototypes  are  deployed  early  -  before  the 
First  Unit  Equipped  date.  Civilian  methodologies  for  early  customer  involvement  will  also  be  examined 
and  contrasted  for  relevance. 
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SOLDIER  INVOLVEMENT  IN  EARLY  EQUIPMENT  PROTOTYPING 


“Show  me  a  man  who  doesn’t  make  mistakes  and  I’ll  show  you  a  man  who 
doesn't  do  anything.”  -  Theodore  Roosevelt 

However, 

“A  man  who  has  committed  a  mistake  and  doesn’t  correct  it,  is  committing 
another  mistake.”  -  Confucius 

Early  soldier  involvement  in  the  materiel  development  process  has  increasing  emphasis 
and  is  a  widely  accepted  “best  practice.”  How  extensively  soldiers  are  involved,  when  they  get 
involved,  and  how  the  process  is  managed  varies  greatly  from  one  system  to  the  next.  Since 
systems  are  different,  the  use  of  soldiers  will  be  different  for  each,  but  an  understanding  of  the 
successes  and  mistakes  of  past  efforts  would  improve  the  process  and  likely  reduce  costs.  The 
core  lessons  cited  in  this  paper  come  from  my  experiences  as  the  Product  Manager  of  the 
Theater  High  Altitude  Area  Defense  (THAAD)  User  Operational  Evaluation  System  (UOES) 
augmented  by  research  into  other  Army  weapon  system  development  and  commercial 
equipment  development  programs.  This  research  paper  addresses  specific  challenges 
associated  with  early  soldier  involvement  in  equipment  prototyping  including:  issues  of  when  to 
involve  soldiers;  how  to  manage  soldier  expectations;  lessons  in  capturing  soldier  input; 
determining  which  soldiers  to  use;  equipment  upgrade  concerns;  contractor  involvement;  safety; 
and  future  directions  including  parallels  with  increasingly  popular  Advanced  Concept 
Technology  Demonstration  programs. 


THEATER  HIGH  ALTITUDE  AREA  DEFENSE  (THAAD)  USER  OPERATIONAL 
EVALUATION  SYSTEM  (UOES) 

Normally  the  Program  Definition/Risk  Reduction  (PD/RR)  phase  of  a  program  yields 
enough  hardware  and  software  to  offer  decision  makers  confidence  that  key  program  risks  have 
been  mitigated  before  entry  into  the  Engineering  and  Manufacturing  Development  (EMD)  phase. 
However,  the  resultant  hardware  and  software  may  or  may  not  have  much  utility  to  a  warfighter. 
During  Desert  Storm,  a  few  systems  were  brought  into  the  war  before  operational  testing  was 
complete,  such  as  Patriot  Advanced  Capability  2  (PAC-2)  and  the  Joint  Surveillance  Target 
Attack  Radar  System  (JSTARS).  With  the  wartime  success  of  those  systems,  key  Defense 
Department  officials  decided  that  if  the  THAAD  system,  which  was  about  to  enter  into  PD/RR, 
was  packaged  a  little  smarter  up  front  in  its  development,  then  the  system’s  utility  in  a  National 
Emergency  would  be  greatly  enhanced.  Congress  embraced  the  concept,  recognizing  the 
critical  need  for  a  system  designed  specifically  to  defend  against  the  growing  proliferation  of 
longer-range  Theater  Ballistic  Missiles  to  be  fielded  as  soon  as  possible.  In  the  Missile  Defense 
Act  of  1991  Congress  expressed  its  desire  for  an  early  capability.  The  Senate  Armed  Services 
Committee  was  even  more  specific  in  1 992  about  the  need  for  the  THAAD  UOES: 

The  committee  recognizes  that  Third  World  ballistic  missile  threats  are  already 
emerging  that  tax  existing  US  TMD  capabilities.  Accordingly,  the  committee 
endorses  current  SDIO  plans  to  develop,  test,  and  field  UOE  prototypes  for  the 
THAAD  system  by  1996.1 

The  THAAD  UOES  concept  satisfies  the  Congressional  language  that  spawned  it  and 
attempts  to  maximize  the  benefits  offered  to  the  materiel  developer  by  an  early  system 
prototype.  The  UOES  provides  early  soldier  input  to  the  materiel  developer  and  has  already 
proven  it  can  save  scarce  future  Research,  Development,  Test  and  Evaluation  (RDTE)  dollars 
by  putting  actual  equipment  in  the  hands  of  soldiers  “early  and  often”  to  solicit  important  design 
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feedback  information.  Examples  of 
changes  already  made  or  planned 
for  the  THAAD  system  include: 
switch  changes,  cable  rerouting, 
durability  enhancements, 
improvements  to  operator/computer 
screen  interfaces,  and  safety 
improvements  that  will  prevent 
injury  to  soldiers  and  equipment 
damage. 

The  UOES  also  provides  warfighters  with  early  operational  assessments  that  will  enable  a  more 
optimized  synergism  of  the  THAAD  system  with  other  battlefield  elements  such  as  the  Patriot 
PAC-3  system  and  Navy  Theater  Missile  Defense  (TMD)  systems.  Finally,  the  UOES 
equipment,  together  with  trained  soldiers,  could  provide  a  critical  interim  Anti-Tactical  Ballistic 
Missile  (ATBM)  capability  to  engage  rapidly  proliferating  longer-range  threat  missiles  in  the 
event  of  a  National  Emergency  -  prior  to  fielding  of  the  objective  THAAD  system. 

...this  is  not  an  abstract  or  a  theoretical  threat  to  our  forces.  The  threat  is  real 
today  and  the  threat  will  only  increase  in  the  years  ahead.  That  is  why  theater 
and  national  missile  defense  development  has  been  a  priority  for  the  Joint  Chiefs 
and  the  unified  commanders.2  [General  Shelton,  the  Chairman  of  the  Joint  Chiefs 
of  Staff] 

WHEN  TO  INVOLVE  USERS/SOLDIERS 

Although  almost  everyone  universally  agrees  that  soldier  input  is  valuable,  the  time 
when  soldiers  are  brought  into  the  development  process  is  widely  varied.  The  contractor 
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Project  Manager  (PM)  for  the  Family  of  Medium  Tactical  Vehicles  (FMTV)  at  Stewart  Stevenson 
agreed  that  soldiers  definitely  provided  valuable  input3.  His  belief  was  that  soldiers  should  be 
involved  as  early  as  the  Operational  Testing  phase.  In  contrast,  the  Navy  PM  who  evaluated  a 
new  replacement  for  the  EA-6B  Prowler  aircraft  involved  soldiers  much  earlier  in  the  process,  at 
the  design  stage,  while  the  drawings  were  still  in  draft  form4.  According  to  the  Navy  PM,  it  was 
very  likely  that  input  he  had  received  from  those  soldiers  would  have  saved  over  two  years  in 
the  development  effort.  However,  the  Navy  PM’s  claims  were  never  proven,  since  the  program 
was  not  funded.  The  current  team  at  FMTV  trucks  has  seen  the  value  of  soldier  involvement 
and  is  now  incorporating  many  modifications  based  on  soldier  feedback  into  the  production 
design.  For  example,  the  shrouds  over  the  taillights  were  strengthened  when  it  was  determined 
that  rather  than  using  the  ladder  intended  for  mounting  and  dismounting  from  the  rear  of  the 
vehicles,  soldiers  simply  jump  into  the  rear  of  the  trucks  by  grabbing  a  side  member  and 
stepping  onto  one  of  the  shrouds.  The  shrouds  had  not  been  designed  to  accommodate  a 
soldier’s  weight.  The  FMTV  team  is  able  to  make  this  and  many  other  smart  changes  to 
vehicles  in  production  or  during  upgrade  operations. 

According  to  many  sources,  including  the  Program  Manager’s  Notebook  issued  at  the  Fort 
Belvoir  Advanced  Program  Management  Course  (APMC),  the  cost  and  impacts  of  changes  are 
significantly  greater  as  the  development  process  goes  on.  Changes  made  during  the  PD/RR 
phase  of  development,  cost  significantly  less  than  changes  made  during  the  EMD  phase. 
Likewise,  changes  in  the  EMD  phase,  made  before  fielding,  cost  significantly  less  than  changes 
made  during  production  when  hard  tooling  must  be  modified.  The  THAAD  team  estimates  that 
nearly  $25  million  in  cost  avoidance  has  been  realized  as  a  result  of  changes  made  early  in  the 
PD/RR  phase  rather  than  later  in  the  development  cycle. 
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Soldiers  get  excited  about  the  chance  to  operate  new  hardware  early,  and  their  feedback 
is  the  most  significant  system  designer  benefit.  There  are  also  significant  challenges  relating  to 
having  them  aboard  early  in  the  design  process.  These  challenges  are  surmountable  if  they  are 
identified  early  and  addressed  in  the  planning  phase.  When  the  novelty  of  being  on  a  new 
system  is  over,  soldiers'  expectations,  training,  and  stabilization  ail  must  be  addressed  and  re¬ 
addressed.  It  is  common  parlance  in  the  field  that  a  good  soldier  can  make  good  equipment 
look  even  better;  a  soldier  also  will  not  hesitate  to  tell  you  that  equipment  is  junk.  Educating 
soldiers  regarding  their  roles  and  the  expected  hardware/software  evolution  is  imperative  to 
maintaining  their  perspective  and  making  the  system  all  it  can  be  throughout  its  development. 

While  equipment  is  still  in  prototype  form  and  soldiers  do  not  yet  have  the  new  system’s 
Military  Occupational  Specialty  (MOS),  the  soldiers  are  promoted  primarily  based  on  mastery  of 
the  tasks  associated  with  operating  the  already  fielded  systems  they  came  from  (the  systems 
from  their  parent  MOS).  If  soldiers  are  involved  with  prototype  equipment  over  a  long  period, 
they  need  to  be  trained  not  just  on  the  prototype  system’s  specific  tasks,  but  also  on  the  tasks 
related  to  their  parent  MOS’s.  Training  can  be  a  formidable  challenge  during  protracted  periods 
of  soldier  involvement  and  requires  flexible  alternative  approaches.  This  may  be  necessary 
because  of  the  potentially  large  segments  of  time  that  the  prototype  equipment  is  back  in  the 
hands  of  the  contractor  for  design,  maintenance,  and  testing.  Computer  based  training  modules 
are  well  liked  and  provide  the  opportunity  for  training  with  very  little  notification. 

Another  issue  of  extended  soldier  involvement  is  that  soldier  turnover  will  be  inevitable 
over  the  years  that  the  system  spends  in  development.  Repeated  soldier  training  is  expensive, 
but  is  essential  to  meeting  the  rotational  demands  of  the  standing  Army  and  to  satisfy  the 
individual  growth  needs  of  the  soldiers  to  avoid  career  progression  problems.  In  summary,  the 
use  of  soldiers  is  likely  to  have  great  benefit,  if  incorporated  early  in  the  process.  Protracted 
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involvement  issues,  such  as  adequate  training,  can  be  mitigated  through  adequate  up-front 
planning. 

MANAGING  SOLDIER  EXPECTATIONS 

Soldiers  are  an  integral  part  of  nearly  all  weapon  systems  and  can  have  significant 
influence  on  apparent  equipment  performance.  There  is  a  saying  that  goes,  if  a  scientist  is 
asked,  “Does  a  system  work?”  he  will  say,  “Yes,  if  it  works  once.”  If  an  engineer  is  asked,  he 
will  say,  “Yes,  if  the  system  works  most  of  the  time.”  If  an  end-user  or  soldier  is  asked,  he  will 
say  “No,  if  the  system  fails  once."  This  adage  was  illustrated  in  the  Ground  Based  Sensor 
(GBS)  Non-developmental  Item  Candidate  Evaluation  (NDICE)  conducted  at  the  Fort  Bliss, 
Texas  Short  Range  Air  Defense  (SHORAD)  test  range  in  spring  1991 .  Seven  radars  competed 
for  a  chance  to  become  the  next  Air  Defense  Artillery  (ADA)  forward  area  sensor  in  repeated 
tests  involving  72  fixed  and  rotary  wing  aircraft  passes  (simulating  friendly  and  hostile  aircraft) 
over  supported  Avenger  weapon  systems  in  each  one-hour  test  period.  The  eventual  winning 
radar  had  an  opportunity  to  win  over  a  year  earlier,  with  no  competition,  but  had  forced  a 
competition  through  its  apparently  poor  performance.  What  made  the  difference  a  year  later 
that  allowed  the  radar  to  go  from  unsatisfactory  to  outstanding?  When  asked,  the  contractor’s 
engineers  replied  that  they  had  made  software  modifications  that  “de-tuned”  the  radar 
somewhat,  resulting  in  slightly  less  sensitivity.  The  radar  still  had  ample  capability  to  meet  all 
ADA  requirements,  but  most  importantly  the  radar  confused  user  soldiers  with  fewer  false 
targets  (a  target  appears  on  the  soldier’s  display  that  is  not  really  there).  When  questioned, 
soldiers  revealed  that  when  they  encountered  high  false  target  rates  they  lost  confidence  in  the 
sensor.  One  of  the  many  factors  that  contributed  to  the  radar’s  final  selection  as  the  winning 
candidate  was  the  soldiers’  confidence  in  the  radar  and  their  willingness  to  rely  on  the  radar 
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data  when  conducting  simulated  Avenger  engagements.  The  winning  radar  has  been  fielded 
and  is  known  today  as  the  highly  successful  Sentinel  radar5. 

In  the  civilian  sector,  the  Callaway  Golf  Company,  with  over  2400  employees,  is  one  of  the 
world’s  leading  producers  of  golf  dubs  and  golf  dub  products.6  Callaway  is  successful  at  least 
partly  because  of  their  concerns  with  managing  the  expectations  of  their  well-defined  customer 
base.  Callaway’s  interface  with  their  customers  is  primarily  through  the  physical  ergonomics  of 
their  equipment.  However,  customer  expectations  are  also  affected  by  the  perceptions  of 
performance  built  through  advertising  and  product  appearance.  Television  announcer  and 
magazine  columnist,  Mr.  David  Feherty  recently  speculated  that  Tiger  Woods  “could  take  any 
bag  [of  golf  dubs]  in  the  room  and  still  find  a  way  to  win.”7  Perhaps  with  Tiger’s  mastery  of  the 
game  that  statement  may  be  true,  but  for  most  golfers  confidence  can  also  be  gained  simply  by 
the  perception  that  they  are  using  a  superior  product. 

Since,  under  the  Rules  of  golf,  mirade  apparatus  isn't  allowed,  most  people  buy 
based  on  perceptions  created  by  the  media.  One  must  be  alert  to  the  fact  that 
when  acquiring  one  brand  of  golf  equipment  that  there  is  very  little  difference  in 
what  that  equipment  can  physically  deliver  over  another.  Often,  when  a  person 
tries  dub  A  versus  dub  B  at  a  test  driving  range,  they  get  excited  about  dub  B 
and  report  how  much  better  it  is  than  A.8 

In  order  to  become  a  leading  producer  of  golf  equipment,  Callaway  has  learned  both  the 
importance  of  superior  technical  performance,  as  well  as  the  importance  of  creating 
psychological  advantages.  Mr.  Tom  Preece,  the  Callaway  Director  of  Research  and 
Development  Operations,  states  that  for  Callaway  to  go  forward  with  a  product  it  must  satisfy 
each  of  three  criteria;  performance,  perception,  and  produdbility.  The  new  product  must 
perform  better  in  some  way  over  an  existing  product.  The  product  must  create  a  desired 
positive  perception,  e.g.,  it  must  look  “forgiving”;  it  should  sound  good  (this  is  acoustically 
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analyzed  in  their  laboratories);  and  it  should  be  consistent  with  brand  awareness  (shared 
features  like  logos  that  are  common  with  other  elements  of  the  product  line).  Finally,  the 
product  must  be  able  to  be  produced  at  consistent  quality  levels  and  at  competitive  cost  levels.9 
Perceptions  are  important. 

A  critical  element  of  the  THAAD  (JOES  development  has  also  been  the  management  of 
expectations.  If  the  expectations  of  the  warfighter,  or  key  materiel  acquisition  decision-makers, 
are  too  high,  then  continued  development  of  the  intended  objective  system  may  be  curtailed. 
For  instance,  in  an  era  of  tightening  RDTE  budgets,  the  prevailing  thought  is  “why  spend  more 
on  the  objective  system  if  the  prototype  is  good  enough?”  The  THAAD  UOES  creates  a  first 
impression  that  it  is  a  highly  capable  end  product,  based  on  well-packaged  system  segments, 
incorporating  many  Military  Off-The-Shelf  (MOTS)  components  and  Government  Furnished 
Equipment  (GFE).  Together  with 
sharp  and  highly  motivated 
soldiers  from  the  1-6  Air  Defense 
Artillery  (ADA)  Brigade,  and  two 
successful  intercept  missions, 
many  visitors  of  the  equipment 
might  justifiably  leave  with  the 
impression  that  the  end-product 
capability  is  now  there. 

Warfighters  may  come  to  rely  too 
heavily  on  the  systems  perceived  capabilities  and  deploy  it  sooner,  at  the  expense  of  the  more 
robust  fielded  system’s  development.  However,  to  be  as  effective  as  warfighters  require, 
hundreds  of  thousands  of  lines  of  software  code  must  still  be  completed.  Also,  upgrades  to  the 
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equipment  must  be  accomplished  to  make  it  sufficiently  rugged  for  soldier  use,  and 
comprehensive  testing  and  evaluation  must  continue  in  order  to  prove  performance. 

Furthermore,  a  dilemma  exists  if  expectations  of  the  prototype  performance  are  too  low. 
The  objective  system  may  be  unfairly  judged  by  the  shortcomings  and  aberrations  typically 
associated  with  developmental  hardware  and  software.  The  THAAD  system  has  made 
remarkable  progress  in  developing  and  integrating  a  complete  weapon  system  including 
launchers,  radars,  battle  management  systems,  and  missiles.  Many  criticisms  have  been  levied 
against  the  missile  segment’s  problems  in  not  hitting  a  target  until  the  seventh  and  eighth 
attempts.  Yet,  there  have  been  repeated  successes  of  all  the  ground  segments  and  continued 
progress  has  been  made  toward  missile  design  robustness  and  producibility.  Many  successful 
weapon  systems  in  the  field  today  required  significantly  more  attempts  and/or  development  time 
early  in  their  prototype  stages.  A  valuable  interim  capability  might  not  be  appropriately  exploited 
in  a  contingency  situation  due  to  a  jaundiced  early  expectation. 


ENHANCED  FIBER  OPTIC  GUIDED  MISSILE  (EFOGM) 

The  use  of  soldiers  in  the  design  of  the  Enhanced  Fiber  Optic  Guided  Missile  (EFOGM)  is 
an  example  of  a  well-planned  evaluation  conducted  early  in  the  development  process.  The  use 
of  soldiers  was  carefully  planned  to  ensure  that  they  understood  the  state  of  performance  of  the 
current  prototype  equipment  and  their  role  in  influencing  the  final  product.  A  potential  downfall 
of  early  soldier  use  is  that  the  soldiers  will  expect  the  prototype  equipment  they  are  working  with 
to  be  representative  of  the  final  product.  As  in  the  Sentinel  radar  case,  soldier  feedback  can 
result  in  relatively  small  changes  that  can  yield  large  changes  in  equipment  performance. 
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Availability  of  soldiers  is  becoming  more  and  more  of  a  premium,  the  draw  down  of  the  Army  to 
current  levels  has  led  to  an  increasing  Operational  Tempo  (OPTEMPO)  and  requires  that 
soldier  involvement  be  well  planned  ahead  of  time  to  ensure  the  maximum  benefit. 

The  EFOGM  early  soldier  evaluation  was  planned  and  conducted  less  than  three  months 
after  a  contract  was  awarded  to  build  a  prototype  system10.  Key  stakeholders,  including  the 
government  EFOGM  project  office,  user  soldiers  from  Fort  Benning,  and  the  equipment’s  prime 
contractor  Raytheon,  were  cooperatively  involved  throughout  the  planning  and  conduct  of  the 
soldier  evaluation.  Preparation  activities  included  an  early  safety  assessment  and  interim  safety 
release  to  cover  the  scope  of  expected  soldier  involvement;  development  of  a  data  collection 
plan;  questionnaire  development;  and  identification  of  environmental  factors  that  might  affect 
performance  (noise,  light,  etc.).  A  pre-evaluation  was  conducted  including  representative 
soldiers  and  a  contractor  system  orientation.  The  lessons  learned  from  the  pre-evaluation  were 
then  used  when  the  evaluation  was  conducted  on  the  following  day.  Each  day’s  testing 
concluded  with  soldier  out-briefs  and  questionnaires  to  capture  perishable  soldier  impressions. 
The  entire  evaluation  required  less  than  one  week  and  fewer  than  20  soldiers,  but  yielded 
significant  design  recommendations  such  as11: 

1 .  Creation  of  channel  guides  for  missile  retaining  pins  to  ease  reload 
operations;  especially  at  night. 

2.  Redesigned  gunner  console  screens  to  be  thinner  (more  room  needed)  and 
to  be  non-glare. 

3.  Added  a  fold-down  step  for  soldiers  to  gain  access  to  roof  racks;  videotape 
revealed  that  soldiers  were  stepping  on  antenna  mounts  to  gain  roof  access 
(similar  to  FMTV  finding  where  soldiers  were  stepping  on  taillight  guards; 
except  that  EFOGM  could  change  the  design  much  earlier,  in  the  prototype 
stage  rather  than  in  the  production  stage,  resulting  in  much  greater  savings). 
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4. 


Changed  filter  locations  to  make  more  accessible  for  required  periodic 
maintenance. 

5.  Added  inside  blackout  curtains  to  reduce  nighttime  detection  from  screen 
glare. 

6.  Changed  missile  canister  lift  points. 

7.  Changed  gunner  screen  displays  to  be  more  intuitive  and  to  guide  gunners 
through  correct  steps. 

CAPTURING  USER  INPUT 

For  extended  periods  of  soldier  involvement,  a  formalized  method  of  capturing  and 
tracking  soldier  input  is  necessary  to  maximize  the  potential  design  benefit.  THAAD 
accomplished  this  task  by  developing  a  soldier  Comment  Tracking  System  (CTS);  by  making 
minor  modifications  to  a  program 
originally  written  for  a  government 
power  company.  The  web-based 
program  collects  soldier  suggestions 
and  allocates  those  inputs  through  a 
project  office  point  of  contact  to  the 
appropriate  government  and/or 
contractor  persons  for  consideration 
and  evaluation.  A  significant  added 
benefit  is  that  soldiers  get  credit  for 
their  inputs  and  can  see  the  progress  of  their  suggestions.  The  increased  visibility  encourages 
even  greater  soldier  participation  in  the  process.  As  a  result  of  early  soldier  involvement  and 
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the  use  of  the  CTS,  hundreds  of  inputs  have  been  formally  collected  with  an  estimated  savings 
of  $24.5  million  in  future  cost  avoidance. 


USER/SOLDIER  REPRESENTATION 

The  Callaway  Golf  Company  selects  focus  groups,  (early  users)  based  on  known 
demographic  differences  that  align  with  their  product  lines,  e.g.,  seniors,  high-handicappers, 
women,  etc.  Similarly,  soldiers  must  be  representative  of  the  soldiers  who  will  eventually  crew 
the  equipment  when  fielded.  Unlike  the  Callaway  example,  where  established  demographic 
sectors  already  exist,  military  applications  involving  new  equipment  often  call  for  an  associated 
new  MOS  group  to  man  or  support  the  system.  These  new  specialties  are  not  likely  to  be 
available  until  very  close  to  the  First  Unit  Equipped  (FUE)  date.  The  THAAD  UOES  solved  this 
challenge  by  using  soldiers  having  an  MOS  that  was  closely  equivalent  to  that  of  the  THAAD 
system.  Therefore,  soldiers  with  Patriot  MOS’s  were  used  in  THAAD's  system-unique  crew 
positions  due  to  the  fact  that  they  had  the  closest  match  to  the  skills  required  during  the  early 
testing  periods. 


PROTOTYPE  UPGRADES 

Once  the  formal  testing  of  prototypes  is  over,  there  is  a  natural  user  desire  to  upgrade 
the  prototype.  THAAD  experienced  desires  from  some  who  stated  that,  “with  the  UOES  you  get 
what  you  get”  contrasted  with  others  who  had  questions  about  why  a  certain  objective  system 
capability  could  not  be  “pulled  back”  earlier  than  was  planned  for  into  the  UOES.  The  decision 
to  completely  or  partially  upgrade  the  prototype  hardware  and/or  software  is  unique  to  each 
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system  developed,  while  it  also  involves  tradeoffs  of  cost,  performance,  equipment  availability, 
and  impacts  to  the  objective  system  schedule. 

The  decision  to  upgrade  the  THAAD  UOES  toward  more  objective  system  capability 
required  a  detailed  analysis.  Many  variables  were  analyzed  including:  the  costs  to  implement 
the  upgrades,  the  expected  remaining  useful  life  of  the  improved  prototype,  the  time  that  the 
prototype  would  be  unavailable  while  it  was  undergoing  upgrade,  the  extent  of  the  testing  that 
would  be  required  for  the  upgraded  system,  and  the  risk  to  the  timelines  and  cost  parameters  of 
the  objective  system.  Members  of  the  THAAD  Project  Office  teamed  to  examine  the  cost, 
schedule,  and  performance  impacts  of  different  options  to  upgrade  the  UOES  system  once  the 
formally  planned  testing  was  completed  and  the  next  development  phase  began.  In  the  final 
analysis,  elements  of  the  THAAD  system  will  likely  undergo  only  very  specifically  agreed-to 
upgrades.  The  THAAD  team  found  that  the  costs  to  upgrade  the  system  were  not  completely 
predictable  nor  were  they  included  in  current  budgets.  If  the  prototype  was  improved,  it  would 
have  less  useful  life  the  more  it  was  upgraded  (it  would  spend  more  time  back  at  the  laboratory 
rather  than  available  fortesting,  soldier  use,  or  contingencies).  The  soldiers  would  get  less  time 
with  the  equipment  the  more  it  was  back  in  the  contractor’s  hand  being  upgraded.  Many  test 
issues  emerged  if  the  configuration  was  allowed  to  continually  evolve  (how  many  changes  could 
occur  before  the  configuration  was  no  longer  consistent  with  the  tested  baseline?).  Most 
importantly,  the  objective  system  timeline  was  put  at  risk  the  more  that  the  system  was 
upgraded.  To  upgrade  the  prototype,  resources  and  time  would  be  drawn  away  from,  and 
adversely  impact,  the  objective  system  development. 

An  independent  review  team,  headed  by  Dr.  Richard  G.  Rhoades,  examined  the  options 
of  upgrading  or  refurbishing  the  radar  elements.  They  found  that  even  for  the  less  expensive 
options,  “These  lower  cost  modifications  all  suffer  from  the  drawback  of  being  'dead  end 
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engineering’  when  viewed  against  the  radar  requirements”  necessary  to  support  an  initial  fielded 
capability.12  Upgrade,  test,  support,  and  dead-end  engineering  costs  are  not  programmed  and 
do  not  support  the  objective  system  development.  For  THAAD,  there  were  no  comprehensive 
system  upgrades  planned,  only  very  specific  changes  were  made  based  on  a  case-by-case 
analysis  of  the  overall  benefits  versus  the  costs  to  implement. 


CONTRACTOR  INVOLVEMENT  PLANNING 

The  contract  must  be  written  to  facilitate  soldier  involvement  The  THAAD  Lockheed 
contract  included  specific  language  that  established  soldier  involvement  up-front  in  the 
contract/proposal  process.  Conversely,  the  THAAD  Radar  Raytheon  contract  did  not  initially 
include  language  to  develop  the  radar  elements  with  soldier  involvement.  The  Raytheon  team 
successfully  incorporated  soldier  involvement  after  many  meetings,  and  a  contract  modification 
was  accomplished,  specifying  the  scope  of  soldier  integration.  Significant  soldier  dissatisfaction 
about  their  inability  to  be  more  involved  with  the  radar  equipment  could  have  been  avoided  if  the 
proper  language  had  been  agreed  upon  prior  to  contract  award.  Proper  contractual  language 
would  have  also  provided  a  much  better  accounting  of  the  projected  cost  impacts,  the  schedule 
requirements,  the  training  support,  and  the  maintenance  requirements  for  the  system.13 

The  Program  Management  Office  (PMO)  and  Prime  Contractor  team  together  will  have 
their  focus  divided  between  the  prototype  and  the  objective  system  development.  From  a 
PMO/contractor  perspective,  the  main  effort  needs  to  be  focused  on  the  objective  system  with 
the  prototype  in  support.  Challenges  include  split  resources  (dollars,  people,  and  time)  for 
running  simulations,  conducting  testing,  and  hardware/software  maintenance.  Simulations  and 
changes  to  hardware  and/or  software  are  run  in  Hardware-in-the-Loop  (HWIL)  facilities.  In 
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order  to  develop  the  objective  system  and  to  fix  problems  with  prototype  systems,  either 
separate  facilities  are  needed,  or  the  same  facility  must  be  capable  of  handling  both  the  old  and 
the  new  hardware/software.  The  people  who  run  those  facilities  need  training.  Likewise, 
maintenance  and  budget  personnel  will  have  the  daunting  task  of  planning  and  executing  repair 
and  maintenance  actions,  on  both  old  and  new  systems,  simultaneously. 


SAFETY  RELEASES 

Involving  soldiers  in  early  equipment  prototyping,  by  definition,  involves  soldier 
interactions  with  hardware  and  software  that  has  been  incompletely  tested.  Close  involvement 
with  TECOM  safety  personnel  can  keep  schedules  and  plans  flowing  smoothly.  Safety  releases 
may  be  required  for  each  equipment  change  in  configuration.  The  THAAD  and  EFOGM  teams 
involved  TECOM  safety  personnel  in  design  meetings,  contractor  test  events,  and  planning 
meetings.  Regular  safety  personnel  involvement  is  essential  to  ensure  that  soldier  safety  is 
never  compromised  and  that  soldiers  and  contractors  are  not  delayed  in  developmental 
activities. 


DELL  COMPUTER  CORPORATION 

The  Dell  Computer  Corporation  is  the  largest  producer  of  personal  computers  in  the 
United  States  with  over  36,500  employees  working  in  over  38  countries.14  Lessons  from  how 
Dell  successfully  allocates  their  resources  by  packaging  largely  existing  subcomponents  to 
satisfy  customer  requirements,  have  parallels  with  the  Army’s  need  to  make  best  use  of  all 
currently  available  hardware  and  software  in  more  innovative  ways.  Dell  pursues  six  phases  of 
product  development  in  what  they  term  “Dell’s  Direct  Business  Model”.  While  different  in 
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number  and  name,  Dell’s 
six  phases  of  product 
development  correlate 
loosely  with  Department  of 
Defense  phases  of 
development.  Dell  is  very 
different  from  military 
developers  and  typical 
hardware  producers  in  the 
relationship  of  resources 


Dell’s  Direct  Business  Model 


Cooperative  Research  &  Development 

The  6  phases  of  product  development 


1 .  Understandhg  what  customers  need 

2.  Defining  the  product 

3.  Desfgningfchocsing  product  components 

4.  Developing  and  integrating  subsystems 

5.  Testing  the  product  for  compatfclfty 

6.  Customizing  and  delvering  the  product 


applied  over  the  development 

FIGURE  4  DELL  BUSINESS  MODEL  (PROPRIETARY)15 

lifecycle.  Dell  places 

relatively  little  emphasis  on  subsystem  research  and  development.  Instead,  Dell  initially  focuses 
its  greatest  resources  on  gaining  a  thorough  understanding  of  customer  requirements,  and  later, 
the  company  focuses  on  customization,  delivery,  and  support. 


Customer  requirements  are  first  identified  through  the  use  of  focus  groups  and  later 
verified  through  pilot  computer  runs  to  ensure  customer  needs  have  been  met.  Once  potential 
user  requirements  are  established,  Dell  is  unique  in  its  ability  to  rapidly  customize  computers  to 
meet  individual  customer  orders,  regardless  of  the  number  of  configuration  and  quantities 
ordered.  Deli  is  also  unique  in  its  ability  to  create  customized,  project  managed,  customer 
support  packages.  Vice  president,  Mr.  Frank  Miller  (MG(R),  USA)  states  the  Dell  customization 
position  as,  “Lead,  follow  and  lose  money,  or  get  out  of  the  business.”  This  model  may  have 
military  applications  in  areas  where  the  requirements  are  relatively  stable  and  well  understood. 
Also,  this  model  applies  significantly  greater  resources  than  applied  under  typical  military 
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applications,  early  in  the  requirements  definition  phase.  Furthermore,  Dell  understands  an  issue 
that  is  also  pertinent  to  the  Army;  specifically,  by  having  a  solid  understanding  of  customer 
requirements  up-front,  they  have  become  a  much  more  efficient  producer  by  wasting  less  on 
follow-on  development  and  redesign. 


ADVANCED  CONCEPT  TECHNOLOGY  DEMONSTRATIONS  (ACTD) 

Advanced  Concept  Technology  Demonstrations  have  been  devised  as  a  means  to  more 
quickly  exploit  emerging  technologies,  or  to  more  quickly  mature  promising  advanced 
technologies  to  solve  military  needs.  Materiel  development  costs  and  timelines  have  been 
under  intense  pressure  to  grow  smaller,  or  to  grow  at  a  slower  rate.  This  pressure  has  spawned 
a  number  of  acquisition  streamlining  initiatives  including  the  ACTDs.  Thus  far,  the  popularity  of 
ACTDs  is  evidenced  by  their  growth  from  26  candidates  in  fiscal  year  2000  to  55  candidates  in 
fiscal  year  2001  (so  far).16  Like  the  THAAD  UOES,  the  hallmarks  of  the  ACTD  program  are  to 
provide  a  rapid  response  to  user  requirements  and  provide  an  interim  capability;  to  develop 
strong  user/operator  involvement  in  order  to  develop  new  tactics,  techniques,  and  procedures; 
and  to  provide  a  fieldable  capability. 

A  key  difference  from  the  THAAD  UOES  is  that  once  the  ACTD  completes  a  nominal  2  to 
4  year  period  of  development  and  integration,  technical  testing,  and  an  assessment  of  the 
military  utility;  there  are  three  general  courses  of  action  that  can  be  pursued.  These  three 
courses  of  action  include  times  when  the  ACTD  should  be  stopped  completely;  the  ACTD 
should  be  modified  or  extended  to  continue  maturation  of  the  technologies;  or  the  ACTD  should 
be  inserted,  where  appropriate,  into  the  traditional  materiel  development  process,  e.g.,  the  EMD 
or  Production  phase  depending  on  demonstrated  technological  maturity  and  urgency  of  need. 
Also,  instead  of  the  nominal  2  years  where  the  ACTDs  are  provided  some  sort  of  limited  fielding 
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for  operational  use,  the  UOES  would  have  been  used  similarly,  but  at  least  2  to  3  times  as  long. 
Currently,  segments  of  the  UOES  are  being  evaluated  for  continued  use  past  2002,  which  is  ten 
years  after  many  of  the  initial  components  were  designed  and  purchased.  Finally,  the  number 
of  soldiers  dedicated  to  the  UOES  was  an  unprecedented  295.  Today,  due  to  the  protracted 
timeline  of  system  development,  the  number  of  soldiers  is  being  reduced  significantly  depending 
on  likely  missions  and  need,  until  a  fielded  system  is  available  for  use. 

Since  ACTDs  are  becoming  a  more  popular  shunt  to  the  traditional  acquisition  process, 
the  2-year  transition  period  during  which  ACTDs  are  provided  to  the  user,  following  the 
development  and  assessment  period,  should  be  and  is  an  area  of  great  emphasis.  The  ACTD 
issues  of  merging  into  the  traditional  acquisition  process  and  determining  the  requisite  reliability 
and  testing  are  accomplished  to  ensure  effectiveness  and  suitability,  while  many  of  the  same 
issues  experienced  by  the  THAAD  UOES  must  also  be  resolved.  In  many  cases  where  the 
need  is  potentially  urgent,  the  user  may  have  some  number  of  soldiers  dedicated  to  operation  of 
the  ACTD  prototype.  Consideration  must  be  given  to  the  number  of  soldiers  involved,  their 
continued  training,  and  the  likely  gap  between  the  funded  2-year  transition  and  the  availability  of 
the  next  generation  of  hardware/software.  The  user  is  not  likely  to  want  to  part  with  the 
equipment  that,  while  prototype,  is  filling  an  urgent  need.  However,  extending  the  useable  life  of 
the  hardware  involves  the  same  tradeoffs  that  THAAD  initially  experienced,  such  as  cost 
increases  over  time,  the  risk  to  the  objective  system  development,  and  the  time  away  from 
soldiers  required  for  upgrade  and/or  maintenance. 

CONCLUSIONS 

The  use  of  soldiers  early  in  the  acquisition  process  has  dear  application  in  the 
development  of  future  army  systems.  Early  user  employment  has  been  shown  to  be  valuable  in 
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different  military  applications  and  is  also  an  industry  best  practice.  Some  of  the  military  lessons- 
leamed  or  industry  best  practices  described  in  this  paper  could  save  resources,  reduce 
developmental  risks,  and  help  to  properly  shape  expectations.  Once  a  program  moves  past  the 
prototype  phase,  the  materiel  developer  must  focus  firmly  on  the  objective  system.  Residual 
ACTD  or  prototype  hardware  can  be  “distractions”  to  development,  and  should  be  managed  by 
a  few  key  persons  on  the  government  and  contractor  team  to  leverage  appropriate  resources, 
on  an  individual  basis,  to  solve  the  issues  at  hand.  The  remainder  of  the  development  team 
should  move  forward  by  implementing  the  prototype  lessons  learned  toward  development  of  the 
objective  system  that  will  meet  the  user’s  requirements.  The  prototypes  should  not  be 
dismissed  with  a  “you  get  what  you  gef  arm  wave.  A  focus  group  can  identify  the  reasonable 
efforts  necessary  to  establish  a  smart  baseline  capability.  That  baseline  capability  should  have 
input  from  warfighters  and  decision  makers  who  will  need  operational  information  at  any 
available  fidelity.  Information  on  reliability  will  be  scant,  software  evolution  must  be  monitored 
and  explained  in  operational  terms,  and  the  impacts  of  commercial  hardware  typically  used  in 
prototype  development  must  be  explained.  Understanding  a  prototype  system’s  limitations  will 
allow  the  warfighter  to  plan  property  and  gain  the  maximum  synergism  of  the  prototype  with 
other  battlefield  systems.  While  clearly  not  intended  to  be  the  objective  system,  prototypes  such 
as  the  THAAD  UOES  may  still  provide  significant  operational  benefits  and  capability  while  the 
objective  system  development  continues. 


RECOMMENDATIONS 

There  is  no  “silver  bullet”  to  precisely  address  how  soldiers  should  be  incorporated  into 
the  materiel  acquisition  process.  However,  some  basic  principles  emerge  repeatedly  regardless 
of  whether  the  user  is  a  soldier  or  a  civilian. 
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1. 


Soldier  involvement  should  occur  as  early  as  possible  in  the  process,  preferably 
before  a  prototype  is  actually  built.  Appropriate  soldier  involvement  should  be  in 
the  design  phase  when  a  drawing  review  is  being  conducted,  as  exhibited  in  the 
EA-6B  follow-on  program;  or  focus  groups  could  be  employed,  as  typically  used 
by  commercial  companies  like  Dell  Computer  and  Callaway  Golf. 

2.  Soldier  involvement  should  be  carefully  planned.  Specific  considerations  of  the 
plans  should  include  selection  of  representative  users  -  whether  it  is  by  an  Army 
breakout  of  representative  MOS’s  or  by  Armed  Forces  Qualification  Test  scores. 
Plans  should  include  specifics  on  how  many  users  will  be  required  (fewer  at  the 
very  early  stages  is  generally  better);  how  long  the  users  will  be  needed  for  the 
evaluation;  and  what  tasks  the  users  will  be  expected  to  perform. 

3.  Expectations  must  be  actively  managed.  All  affected  stakeholders  should  be 
identified,  e.g.,  users,  testers,  Congress.  During  test  periods,  explanations 
should  be  provided  describing  the  purposes  of  the  evaluation,  along  with  a 
description  of  the  prototype  nature  of  the  hardware  and/or  software  being 
evaluated. 

4.  User  inputs  should  be  formalized.  Formal  collection  of  inputs  ensures  maximum 
benefits  from  early  user  involvement.  It  also  helps  to  later  justify  the  costs 
associated  with  early  user  involvement  against  the  cost  savings  realized  from 
user  suggestions.  A  long-term  collection  like  the  THAAD  CTS  is  appropriate  for 
protracted  user  involvement,  or  a  short-term  collection  like  the  EFOGM  is 
appropriate  for  limited  duration  user  involvement. 

5.  User  involvement  should  be  repeated  wherever  feasible,  throughout  the 
development  process.  Different  users  are  likely  to  generate  different  design 
inputs  -  even  if  all  users  are  using  the  same  equipment  baseline.  When  baseline 
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changes  do  occur,  early  user  input  can  be  used  as  a  cost  effective  means  to 
validate  or  refute  the  value  of  the  change. 
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ENDNOTES 


1  Senate  Armed  Services  Committee  Conference  Report.  7  July  1992. 
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